Long Short-Term Memory

نویسندگان

  • Sepp Hochreiter
  • Jürgen Schmidhuber
چکیده

Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient-based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O(1). Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

The effect of intrahippocampal microinjection of Naloxone on short –term and long-term memory in adult male rats

Introduction:The hippocampus is one for the major centers of learning and memory. Role of the opioid system has been investigated and on the other hand receptors related to this system such as mu-opioid receptors (MOR) are extended in the hippocampus. In this study the effect of Naloxone administration as a mu opioid receptor antagonist on passive avoidance memory in adult male rats was i...

متن کامل

Effects of exercise on spatial memory deficits induced by nucleus basalis magnocellularis lesions

Introduction: Previous studies have shown that exercise enhances cognitive and functional capacities in patients with Alzheimer's disease (AD). In this study, we investigated the effect of long-term (60 days) and short- term (10 days) exercise on the spatial memory deficits in an animal model of AD. Methods: Fifty male rats were divided into 5 groups 1) intact, 2) sham, 3) sham-Alzheimer 4) ...

متن کامل

Effect of Short and Long-Term Treatment with Omega-3 Fatty Acids on Scopolamine-Induced Amnesia

   Two omega-3 fatty acids including docosahexaenoic acid (DHA) and eicosapentaenoic acid (EPA) are essential for the physiologic function of neuronal cell membrane. Normal function of neuronal cell membrane requires appropriate composition of fatty in its structure. Present study was designed to compare the effect of short-term and long-term pretreatment with omega-3 fatty acids on scopolamine...

متن کامل

P7: The Roles of Long-Term Memory on the Organization of the Knowledge for Educators

Modern neuroscientific research help to solve the impotent challenge in curriculum design and teaching for enhancing students’ ability to organize information in a way that makes it efficient in response to an appropriate context such as problem solving and critical thinking via knowing about the mechanism of different type of memories especially long term memory. At first, we should to c...

متن کامل

Effect of Short and Long-Term Treatment with Omega-3 Fatty Acids on Scopolamine-Induced Amnesia

   Two omega-3 fatty acids including docosahexaenoic acid (DHA) and eicosapentaenoic acid (EPA) are essential for the physiologic function of neuronal cell membrane. Normal function of neuronal cell membrane requires appropriate composition of fatty in its structure. Present study was designed to compare the effect of short-term and long-term pretreatment with omega-3 fatty acids on scopolamine...

متن کامل

Effects of fresh, aged and cooked garlic extracts on short- and long-term memory in diabetic rats

Objective: The present study was hypothesized to investigate the beneficial effects of fresh, aged, and cooked garlic extracts on blood glucose and memory of diabetic rats induced by streptozocine (STZ). Material and Methods: Diabetes was induced by an intraperitoneal injection of STZ (60 mg/kg body weight). An oral dose of 1000 mg/kg of each garlic extract was given daily for 4 weeks after dia...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • Neural computation

دوره 9 8  شماره 

صفحات  -

تاریخ انتشار 1997